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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments filed 12/21/2009 have been fully considered but they are 
not persuasive. 

2. Re claims 1,16, and 23, the Applicant contends that the prior art fails to teach or 
suggest calculating a rate of change based on at least three frames, but instead only 
refer to calculating motion vectors between two frames. However, the Examiner 
respectfully disagrees. Zhang discloses that the motion vectors are extracted from each 
frame of the video sequence (Zhang: column 9, lines 26-27), meaning that motion is 
determined between more than just two frames. 

3. Re claims 1,16, and 23, the Applicant also contends that the prior art fails to 
teach or suggest determining a distance of a moved pixel from a current frame to a first 
frame, then determining a distance of the moved pixel from the first frame to a second 
frame, and then calculating a difference between the two distances to determine a rate 
of change of visual content. However, the Examiner respectfully disagrees. Zhang 
discloses that a perceived motion energy (PME) is calculated (Zhang: Fig. 3), wherein 
the PME takes into account the average magnitude of the motion vectors to represent 
the rate of change (Zhang: column 9, lines 39-49). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1, 3, 4, 6, 9-11, 13-16, 18, 19, 21, 23, 25, 26, and 29 are rejected under 
35 U.S.C. 103(a) as being unpatentable over Zhang et al. (US 7027513 B2). 

Re claim 1, Zhang discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: using a processor to determine a rate of change of 
visual content of each current frame from a corresponding reference frame, each of said 
current frame and said reference frame being comprised in said sequence of image 
frames (Zhang: column 9, lines 10-24), wherein said rate of change represents a 
difference of a first value and a second value, said first value representing a change of 
visual content of a current frame compared to a first frame, said second value 
representing a change of visual content of said first frame compared to a second frame, 
wherein said second frame is a reference frame for said first frame and said first frame 
is a reference frame for said current frame (Zhang: column 9, lines 25-49); and using 
the processor to select said current frame as a corresponding one of a set of potential 
video frames if said rate exceeds a corresponding first threshold value, wherein said 
plurality of key video frames are selected from said set of potential video frames 
(Zhang: column 9, lines 50-60 and column 10, line 64-column 11, line 18), wherein said 
determining and said selecting are repeated for each of said sequence of image frames 
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as said current frame to form said set of potential video frames (Zhang: column 9, lines 
25-27, vectors extracted from each frame); wherein said determining comprises: 
determining a displacement magnitude of each moved pixel of said current frame 
compared to the position in said first frame and of said first frame compared to the 
position in said second frame (Zhang: column 9, lines 27-30); and computing a first 
representative magnitude of said displacement magnitude for said moved pixels of said 
current frame compared to said first frame, and a second representative magnitude of 
said displacement magnitude for said moved pixels of said first frame compared to said 
second frame (Zhang: column 9, lines 25-30, vectors extracted from each frame), 
wherein said first value and said second value respectively equal said first 
representative magnitude and said second representative magnitude such that said rate 
is computed as a difference of said first representative magnitude and said second 
representative magnitude (Zhang: column 9, lines 25-38, cumulative calculation). 

Zhang discloses that motion vectors are extracted from each frame (Zhang: 
column 9, lines 25-27), but Zhang does not explicitly disclose that each of the respective 
first frame and the respective second frame are different for different ones of the current 
frame in said sequence of image frames. However, one of ordinary skill in the art at the 
time of the invention would have found it obvious that as the current frame changes 
sequentially, the reference frame for calculating motion vectors changes also, because 
if the reference frame were not to change the motion data would progressively 
degenerate in accuracy and relevance. Therefore, by stating that each frame is 
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analyzed, Zhang indicates changing the reference frame along with the current frame 
and maintaining a relative positional relationship, as is well known in the art. 

Re claim 3, Zhang discloses that said first representative magnitude for said 
current frame equals an average of motion energy vector magnitudes of said moved 
pixels of said current frame in comparison with corresponding pixels of said first frame 
(Zhang: column 9, lines 25-38, the average magnitude of the motion vectors for a 
particular frame is calculated; column 9, lines 39-49, and the average magnitude is used 
to calculate the perceived motion energy). 

Re claim 4, Zhang discloses that said first threshold value is the same for all of 
said current frames, said selecting further comprises: including said current frame in 
said set of potential video frames only if said first representative magnitude exceeds a 
second threshold (Zhang: column 10, lines 28-33, the triangles must be larger than a 
minimum size); and including only those of said set of potential video frames, which 
exceed said first threshold, in said plurality of key video frames (Zhang: column 10, lines 
37-48, triangles not meeting size constraints are resized). 

Re claim 6, Zhang discloses identifying a plurality of active pixels in said current 
frame, wherein a pixel is considered an active pixel if a corresponding displacement 
magnitude is outside of a range, wherein only said plurality of active pixels are used by 
said computing (Zhang: column 9, lines 33-38, dominant motion direction percentage 
calculation). 

Re claim 9, Zhang discloses enabling a user to specify one of a plurality of key 
video frames, wherein said plurality of key video frames are selected by said selecting 
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(Zhang: Fig. 1, element 120; column 5, lines 43- 44, indicate that the user may use the 
key frames to select the desired section of the video for display); and displaying said 
specified one of said plurality of key video frames (Zhang: Fig. 1 , element 1 20; column 
5, -lines 43-44, indicate that the user may use the key frames to select the desired 
section of the video for display). 

Re claim 10, Zhang discloses displaying a prior key video frame and a next key 
video frame in relation to said specified one of said plurality of key video frames, 
wherein said prior key video frame and said next key video frame are comprised in said 
plurality of key video frames (Zhang: Fig. 1 , element 120, key frames are displayed in a 
sequence). 

Re claim 11, Zhang discloses generating a display indicating the manner in 
which said plurality of key video frames are interspersed in said sequence of image 
frames, wherein said enabling is based on said display (Zhang: Fig. 12, the key frames 
are shown along with their corresponding frame numbers). 

Re claim 13, Zhang discloses generating a display listing said plurality of key 
video frames, wherein said enabling is based on said display (Zhang: Fig. 1, element 
120; column 5, lines 43-44). 

Re claim 14, Zhang discloses that motion vectors are extracted from each frame 
(Zhang: column 9, lines 25-27), but Zhang does not explicitly disclose that the 
corresponding first frame and the corresponding second frame are selected at a same 
respective relative position in comparison to the position of the current frame such that 
each of the respective first frame and the respective second frame are different for 
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different current frame. However, one of ordinary skill in the art at the time of the 
invention would have found it obvious that as the current frame changes sequentially, 
the reference frame for calculating motion vectors changes also, because if the 
reference frame were not to change the motion data would progressively degenerate in 
accuracy and relevance. Therefore, by stating that each frame is analyzed, Zhang 
indicates changing the reference frame along with the current frame and maintaining a 
relative positional relationship, as is well known in the art. 

Re claim 15, Zhang discloses that said current frame, said first frame and said 
second frame are in consecutive successive positions in said sequence of frames 
(Zhang: Figs. 9A, 9B, 10A, and 10B, PME graphs go in order of frame number, 
indicating that calculations are performed sequentially). 

Re claim 16, arguments analogous to those presented for claim 1 are applicable 
to claim 16, and, therefore, claim 16 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 18 has been analyzed and rejected with respect to claim 3 above. 

Claim 19 has been analyzed and rejected with respect to claim 4 above. 

Claim 21 has been analyzed and rejected with respect to claim 6 above. 

Re claim 23, arguments analogous to those presented for claim 1 are applicable 
to claim 23, and, therefore, claim 23 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 25 has been analyzed and rejected with respect to claim 3 above. 
Claim 26 has been analyzed and rejected with respect to claim 4 above. 
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Re claim 29, Zhang discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: using a processor to receive receiving said 
sequence of frames of a same scene/area of interest according to a sequential order 
(Zhang: column 9, lines 18-24); using the processor to choose choosing one of said 
sequence of image frames as a current frame, a first frame being before said current 
flame and a second frame being before said first frame according to said sequential 
order, said first frame being at a first relative position in relation to said first frame in said 
sequential order and said second flame being at a second relative position in relation to 
said first frame in said sequential order (Zhang: column 9, lines 25-38, motion vectors 
extracted from each frame of the sequence); using the processor to calculate 
calculating a displacement magnitude difference of a first value and a second value, 
said first value representing a measure of a displacement magnitude change of visual 
content of said current frame compared to said first frame, and said second value 
representing a measure of a displacement magnitude change of visual content of said 
first frame compared to a second frame (Zhang: column 9, lines 25-38, motion vectors 
extracted from each frame of the sequence); using the processor to select selecting 
said current frame as a corresponding one of said plurality of key video frames if said 
difference exceeds a first threshold value and first value exceeds a second threshold 
value (Zhang: column 9, lines 50-60 and column 10, line 64-column 11, line 18); and 
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using the processor to repeat repeating said calculating and said selecting after 
choosing each of said sequence of image frames as said current frame to form said 
plurality of key video frames (Zhang: column 9, lines 25-27, vectors extracted from each 
frame. 

Zhang discloses that motion vectors are extracted from each frame (Zhang: 
column 9, lines 25-27), but Zhang does not explicitly disclose that each of the respective 
first frame and the respective second frame are different for different ones of the current 
frames, and are respectively determined based on the same first relative position and 
said second relative position in reference to the corresponding current frame. However, 
one of ordinary skill in the art at the time of the invention would have found it obvious 
that as the current frame changes sequentially, the reference frame for calculating 
motion vectors changes also, because if the reference frame were not to change the 
motion data would progressively degenerate in accuracy and relevance. Therefore, by 
stating that each frame is analyzed, Zhang indicates changing the reference frame 
along with the current frame and maintaining a relative positional relationship, as is well 
known in the art. 

6. Claims 5, 7, 8, 20, 22, and 27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Zhang et al. (US 7027513 B2) in view of Ma et al. (US 
20040088723 A1). 

Re claim 5, Zhang discloses a majority of the features of claim 5, as discussed 
above in claims 1-4, but does not specifically disclose that the first threshold and the 
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second threshold are adjusted dynamically to ensure that a desired number of frames 
are selected as key video frames in a specified duration. However, Ma discloses a 
method for generating a video summary, where a binarization threshold is estimated in 
an adaptive manner (Ma: paragraph [0081]) when analyzing a video sequence for 
selecting key frames. Since Zhang and Ma relate to selecting key frames in a video 
sequence, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the attention modeling of Ma with the key frame selection method of 
Zhang in order to create a representative video summary consisting of very short video 
clips, which contain the video immediately preceding and immediately following key 
frames that have been selected (Ma: Fig. 20). 

Re claim 7, the combined method of Zhang and Ma discloses that said range set 
by a distance of two times the variance from the mean of a distribution (Ma: paragraphs 
[0081]-[0082], the threshold is a function of the variance). 

Re claim 8, the combined method of Zhang and Ma discloses that said 
representative magnitude comprises an average of said active pixels (Ma: equation (12) 
and paragraph [0084]). 

Claim 20 has been analyzed and rejected with respect to claim 5 above. 

Claim 22 has been analyzed and rejected with respect to claim 7 above. 

Claim 27 has been analyzed .and rejected with respect to claim 5 above. 

7. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zhang et al. (US 7027513 B2) in view of Sull et al. (US 20060064716 A1). 
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Re claim 12, Zhang discloses a majority of the features of claim 12, as 
discussed, but does not specifically disclose that the display comprises a pie chart. 
However, Sull discloses techniques for navigating multiple video streams, where 
textual/visual information, such as a pie chart, may be displayed along with poster- 
thumbnails on the video selection screen for the user interface (Sull: paragraph [0307]). 
Since Zhang and Sull all relate to representative images for video sequences, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the textual information of Sull with the key frame selection of Zhang in order to 
provide the user with more information, such as date and time of broadcast (Sull: 
paragraph [0307]), for improving the user's ability to quickly find the desired video 
segment (Zhang: column 5, lines 44-45). 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

a. Feature based hierarchical video segmentation; Bozdagi et al. (US 
6493042 B1 ) 

b. Method of selecting key-frames from a video sequence; Wilf et al. (US 
7184100 B1) 

c. System for automatic video segmentation and key frame extraction for 
video sequences having both sharp and gradual transitions; Zhang et al. (US 
5635982 A) 
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9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 



Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
(571)270-1199. The examiner can normally be reached on Monday-Friday (8:30 AM- 
5:00 PM). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha D. Banks-Harold can be reached on 571-272-7905. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Marsha D. Banks-Harold/ 

Supervisory Patent Examiner, Art Unit 2621 

/Christopher Findley/ 



